Learning Category- and Instance-Aware Pixel Embedding for Fast Panoptic Segmentation

Panoptic segmentation (PS) is a complex scene understanding task that requires providing high-quality for both thing objects and stuff regions. Previous methods handle these two classes with semantic instance modules separately, following heuristic fusion or additional to resolve the conflicts between outputs. This work simplifies this pipeline of PS by consistently modeling novel framework, which extends detection model an extra module predict category- instance-aware pixel embedding (CIAE). CIAE pixel-wise feature encodes semantic-classification instance-distinction information. At inference process, results are simply derived assigning each detected class according learned embedding. Our method not only demonstrates fast speed but also first one-stage achieve comparable performance two-stage on challenging COCO benchmark.